Skip to content

snap census: rows for the nextTick queue, scheduled resolutions and timers pending at the cut (instrumented builds only) - #14466

Merged
proggeramlug merged 34 commits into
mainfrom
snap/census-k4
Nov 11, 2026
Merged

proggeramlug merged 34 commits into
mainfrom
snap/census-k4

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

What

The instrumented snapshot census (builds with PERRY_RUNTIME_INSTRUMENTS=1) names three more kinds of work that is still pending at the cut, next to the existing job_queue row:

row policy entry
tick_queue image tick:<owner init>:<callback code>
scheduled_resolves image resolve:<owner init>:0
timer_queue STOP <timeout|interval|immediate|promise|native>:<owner init>:<callback code>:<delay ms>:<repeat>

Each row carries init=<cut>;pending=N, the first 64 entries and +N for the rest, or busy when the store is borrowed at that moment. The rows share the existing PERRY_SNAP_JOBQ=0 switch with job_queue. Owners are stamped only in the instrumented build.

Why

The census must say what a snapshot taken at the cut would have to carry or stop on. Pending microtasks were covered (#14449); pending process.nextTick callbacks, scheduled promise resolutions and timers/immediates were not, so a program with one of those at the cut looked clean.

Verification

check result
cargo check --workspace --exclude perry-ui-gtk4 rc=0
perry-runtime --lib snap (production cfg, merged head) 64 passed, 0 failed
instrumented snap_pending tests (4 new + the job test) 5 passed
fixture snapk2b/f14q (one tick, one resolve, timeout + interval + immediate at the cut) PASS: tick_queue pending=1, scheduled_resolves pending=1, timer_queue pending=3 policy=STOP
fixture snapk2b/f12 (job_queue) PASS, unchanged

Sabotage (each reader made to return an empty list): pending_ticks_name_owner_and_callback, scheduled_resolves_name_owner and pending_timers_name_kind_owner_callback_delay_and_repeat each fail.

Production builds: hello is byte-identical to the base (md5 fc3da781…, .text 1270). commander has identical section sizes and an identical instruction stream to the base once addresses are stripped (objdump, .text 9,936,062 on both); the raw md5 differs through build-path-derived addresses only.

Ralph Küpper added 30 commits November 6, 2026 18:20
… (SNAP25)

Instrumented builds only (cfg(perry_gc_instruments)); the production build
is unchanged. PERRY_SNAP25=0 restores the SNAP23 rows.

- Native rows carry the captured return pcs.
- Funnels (exec_path, stack_maps, stack_bounds, parse, parallelism, gc,
  exec_argv) tag the syscalls and native rows issued inside them; the GC
  heap budget's first computation is in the gc funnel.
- navigator.hardwareConcurrency is an input call; env reads inside the
  exec_argv funnel are program inputs (guard members, never knobs).
- unsetenv/setenv rows record whether the name was set; timer, uptime,
  console.time and GC-stats clocks are input rows.
- Every Error stack capture records whether its pcs lie in the executable
  text.
- Census compiles mark each process.argv read: length, constant index
  (Number or Integer literal) or escape.
- The fault handler brings a tracked page whose policy is open but whose
  protection is still an earlier bulk protect back to its policy instead
  of treating the fault as foreign.
# Conflicts:
#	crates/perry-runtime/src/process/env_misc.rs
…P-reach rows; fixtures F1 F2 F3 F10 (instr-only)
…y words, owned storage in the (c) scan (instr-only)

- F1/F2/F3/F10 call gc() before the output, so the boundary report and the
  K2a rows are written; checkk2.py fails "no boundary row" first.
- on_trap applies protection after every store of the step is recorded;
  note_access returns whether the first-write watch ended and on_segv calls
  end_watch(). A nested fault inside on_trap reports 0xdead instead of
  dying on the blocked SIGTRAP.
- P-entry (a) uses its own word set `ew` (last writer is an entry); (a) is
  a conflict rule and is read from the PERRY_SNAP22_WATCH=0 run.
- The p_reach row follows the init_enter event (exact Ir accounting join).
- (c) scan: an entry birth reached through an owned-storage edge of a
  non-entry holder (ObjectHeader.meta; ObjectMeta spill, overflow,
  elements, expando, owned_state) is the holder's representation, not an
  entry value: no target, scanned as the holder's words. F2 named the
  global's meta record (born by the runtime, holding the spill that the
  entry's globalThis.k grew) as a (c) holder; checkk2 F2 now allows only
  the box.
- held rows name holder_type and holder_seq.
# Conflicts:
#	crates/perry-runtime/src/navigator.rs
…e and job-queue rows, runtime-pc STOP, worker-flag STOP
…lours, TERM=dumb does not)

util.styleText with the default stream check now follows node's
shouldColorize(process.stdout): with FORCE_COLOR set the text is styled when
getColorDepth() > 2, whatever stdout is; otherwise stdout must be a tty and
getColorDepth() > 2. The old check returned false for FORCE_COLOR="" (node
colours: depth 16) and coloured a tty with TERM=dumb or no TERM (node: depth 2).

The decision is tty::stdout_style_bit(), over the same colour-depth lookup as
process.stdout.getColorDepth().

tests/parity/run_style_text_color_tests.sh compares stdout with node for
FORCE_COLOR unset/""/0/1/2/3/false, NO_COLOR, NODE_DISABLE_COLORS, TERM=dumb,
TERM=xterm-256color, no TERM and two FORCE_COLOR combinations, each under a pty
and through a pipe.
tty.rs: stdout_style_bit is the style fix plus the instrumented StyleBit funnel and stdout_style_bit row (cfg perry_gc_instruments).
…g text

scripts/snap_inputs/checkk2b.py and fixtures/snapk2b/{f3n,f4,f5,f6,f7,f8,
f12,f13,fw,f2c}: each fixture is red with its rule knob off and green with
it on (f8 confirms its row only).

compare_literal read a heap string through the K2a value encoding
("string:<hex>") but a short string as its text, so equal text in the two
representations compared unequal and the row's literal held the encoding.
string_text now returns the text for both.

f12 queues a promise job at module scope that is still pending when the
next module starts (node order: n sees m unset); a TLA module does not
reach that state in perry, which runs it to completion before its
sibling. f13 makes its Error in a recursive function at module scope: a
module init body is not JS text, so an Error made there captures no frame;
the checker now requires a non-empty capture.
The compare_literal census hook kept a thread-local list of compare
sites keyed by Expr address. A compare whose numeric-literal or typeof
operand is lowered without lower_expr never completed its entry; once a
codegen thread moved to the next unit, an expression at the freed
literal's address emitted js_snap_compare_literal with the earlier
function's register (Supabase: precise_roots native-root lowering panic
in an unrelated closure).

expr::compare::lower now emits the hook from its own re-read operand
values, right where it compares them. Fast paths (typeof, numeric
literal, chained and typed compares) emit no row: the read stays a
value member. No site list remains. The new test compiles a module of
fast-path compares, then a module with no compare, and asserts the
second emits no hook (33 stray hooks on the old code).
The process.env / process.argv census arms looked their parent up in
site lists keyed by node address, so a later module whose nodes reused
freed addresses read stale entries. The lists are gone: FnCtx keeps the
two innermost expressions being lowered (installed by lower_expr and by
a module-global let, restored on every return), and each read
classifies itself against its own parent. The scalar-replacement paths
of lower_let lower literal operands with the literal installed as
their parent. Test env_argv_kinds_never_cross_reads.
…side it does not walk the stack

A census native event (clock_gettime from mimalloc's arena free, reached
from decode_fde_row inside find_handler) captured its stack through the
same eh_walker, whose lock the thread already held: the Supabase census
run hung at the ws buffer-util throw (init 272). locked_walker returns a
WalkerLock that holds the census Unwind funnel in the instrumented cohort;
native_event skips the capture inside it and the row carries
funnel=unwind. Production builds are unchanged (the funnel field is
instrumented-only).
…ode, queueing init)

The K3 Supabase census stops at init 1033 on job_queue (pending=1 at every
later boundary), but the row only counted the jobs. Each row now lists the
queued jobs in queue order as kind:owner:code: the Task variant, the init
that queued the job (the snapshot's snap_init, set at enqueue) and the
callback's JsFunctionInfo code address (0 for a reaction without a callback).
Read from the task itself; instrumented runtime only.
…sus-k3c

# Conflicts:
#	crates/perry-codegen/src/expr/compare.rs
#	crates/perry-codegen/src/expr/mod.rs
…imers at the cut

At every candidate cut the census reads the run queues beside TASK_QUEUE from
their own stores, one row per non-empty store: tick_queue (tick:owner:code)
and scheduled_resolves (resolve:owner:0), imaged with the task queue by the
same FIFO-prefix rule (SNAP_DESIGN 2.10.21), and timer_queue
(kind:owner:code:delay_ms:repeat for timeouts, intervals, immediates, promise
timers and native completions), a blocker (policy=STOP): it waits on the host
clock or host I/O.

The owner is the init that queued the entry, carried by the entry itself:
timer entries stamp AsyncContextSnapshot::snap_init when built, and a
scheduled resolution carries it as a third tuple field. Both only in the
instrumented cohort; production types and code are unchanged.

Fixture snapk2b/f14q; unit tests for each store and the empty case.
@proggeramlug
proggeramlug marked this pull request as ready for review November 11, 2026 15:16
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Explain Change →

Pre-merge checks: Inconclusive 2

Failed Failed checks: 2 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Description check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@proggeramlug
proggeramlug merged commit 89fd729 into main Nov 11, 2026
95 of 101 checks passed
@proggeramlug
proggeramlug deleted the snap/census-k4 branch November 11, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant